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EVALUATING TEACHERS 

J. Dale Burnett 
f Faculty of Education, Queen's UniVv^|^ty. . 
Kingston, Ontario 

» 

The title strikes terror In the hearts of teachers and \ 
evaluators alike, Brywnie points, professional recommendations, merit 
pay, accountability, promotions, job security, student power, complaints, 
ellabillty, attendance, departmental exams, extracurricular activities, 
cafeteria duty, tldyness, valldltyl organization, personal appearance, 
gain scor^ and so on. Who 1&, to say what ^preferences lurks In the 
hearts of^menl The Shadow may knoi&; - but that Isn't much help^ to 



the rest of us mortis, 

A 

;l^Qh with the toplc'I The| present paper will summarize a tiumber 



. m 

of current approaches to teacher evaluation, review the progress to 



-as much fun as nostalgia, out 



date, and provide a suggestion for future consideration. Perhaps not 

certainly just as Importai^t. 

/ ^ 

CURRENT APPROACHES 
In recent years there has been a tendency among evaluators 
t\D divide educational evaluation into two categories: process and 
product. Historically the epphasls has been on the process component 
- the school Inspector or principal observing ^a teacher in the class- - 

/room and then providing a brief report on the teacher's strengths 
and weaknesses. However, lately, there has been an increasing inter^^^^ 

focusing on what a student can dp as a result of instruction - the 

« 

product dimension. 



A listing of current approaches to teacher evaluation should 
include pHfe following: - 

il*. Examinatiqn of stqd'ent gain scores on standardic?ed tests 

2. . Analysis of typical. student improvement 

3. '> Teaching Performance Testing % 

4. Teacher skill testing 

5. 'Administrator or peer group observations and rating^ 

6. Student observations and ratings. \ ^ 
^*he first three approaches tend to emphasize the product dimension 

o^ teaching whereas ther latter three focus more on the process of r 
^teafHing^ A bri^f description of the six approaches is appropriate 
since some of the labels have yfet ,to, receive wide acceptance. 

1. Examination of student gain scores on standardized tests. 
This approach received its impetus from the performance con- 
tracting movement in the United States. Essentially ^he procedure 
involves administering a conpercially available test in the relevant 
subject area, for;, example r readlngXin September and then again in 
June. The improv6ment in student scores is taken as a measure of 
teacher effectiveness. Glass (1974) provides a scathing review of 
such practice, citing such weakn^s^es as irrelevance of many items, 
non'random assignment of studelits^t;o classes, unreliability of the 
improvement index and high administration costs. 

2. Analysis 6f typical student improvement. 
This is really a variant of the first approach. A pre-test 

is administered at the beginning of terra to all the students in a 
district for a specific grade and subject matter. At the end of term 
a post-test is given to the same students and averages for both tests 
for each clase are computed. A point (pr6-test average, post-test 
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average) is plotted for each class and a regression line of best fit 
Is determined to represent the "typical" relationship between the 
two i tests over all of the classes In the district. Teachers whose 

I ■ • < . 

cla|sses. fall above the line are said to do better than average and 

thos^ who f^il below the line do poorer than average. It Is Important 

to recognize that the nature of the statistical procedure ensures that 

approximately half of the teachers must perform below the average. 

Most of Glass' earlier criticisms would seem to apply equally well 

to this approach, 
ft 

3. Teaching Performance Testing. f 

Very briefly, ^the Idea underlying this approach Is to have .^^ 

^: 

the teacher. provide a short lesscm on a novel topic and then to measure 

^. I • ^ ^ ^ I 

hls/he|r effectiveness by noting how well the stiidpnts do on an achieve- 

ment tast. more ^detailed descrljptlon (Popham, 1973) , a report on 

the use of the procedure (Popham,* i971) and a critique of the procedure 

(Glass|, 1974) all provide addltiqijia^ liifqrmatlon on this approach. 

4. Teacher skill testing. 



The teacher being evaluated Is presented wibh^^a typical^, class- 
room ptroblem and Is asked to eltheti say or write what her^s^ution 
would be. The adequacy of the solutl^on Is used to Infer the teafeh^r's ^ 
effectiveness. This ^approach Is a cotiunon component of many teacher 
education programs but Is unlikely ' to receive wide acceptance In the 
raor^-.-fMra'cTlcal envlx:onmQnt of a school system. 

5. ' Admlnlstr^^tor or peer group observations and ratings. 

V 

Ab ifnentloned earlier, thls.--apin:oach has the longest history. 
Thus In spite- of* some statistical difficulties, notably reliability 
and to some extent validity, this approach- must possess a satisfactory. 



although not a desirable, degree of utility. The main problem appears 



/ 



to centjfer on the issue of what 
problem have focused on imposir 



to look for. Attempts to minimize this 
g a greater degree of structure on 



the observation procedure. Special forms, and even training, are often 



required before becoming an observer 



&• Student^ observatio 



IS and ratings. 

^ ■ ■ I ■ ■ 

■ Although the procedure is very similar to the previous approach*, 
^the critical difference is in the natur^/^f the observer. In this case 
it is" the recipient, the consumet, of the actual instruction. While 
suffering frbm the same technical difficulties as other observation 
systems, this appro aiali^ has gained in popularity in recent years. Per- 
haps the main dif f iculty^^t;h this approach oc;purs after the data has 
been collected and summarized, namely that of i^iterpretfation. For 
example, what UmportainrB--aliould one ^tt^^ph^io a relatively low rating 
on a stt^dent attitude scald? * ' ^ 

REVIEW OF TEACHER EVALUATION 
The topic of teacher evaluation^ is not a new one. Williams 
(1971) has provided a brief review stretching from about 500 BC in , 
Greece to 1970 AD in North America. After 2500 years of attention, 
recent summaries indicate that little progress ha s"" been achieved: 
**Despite all of our efforts,^ we apparently have no generally accepted 
conceptual system, psychological or otherwise, by which either to 
formulate or to identify the skills of teaching (Smith, 1971, »p. 3)." 
''It is possible that the patterns of effective teaching for different 
ends are so idiosyncratic that they will never be isolated; it is 
possible that studying teaching in natural settings in unproductive 
because the settings are not functional for the desired outcomes... 
At the moment there has not been enough research to make any firm 
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statement about any of these concerns (Rosenshlne and Furst, 1973, 
p.175)." • 

"Teacher educators err when they promote teaching skills that are 
approximately consistent with scientific conclusions as if these skills 
were cert ain, confirmed answers about how a teacher should proceed to 
' ^ effect desirable consequences in le^rnerJl Instead, such fifkills should ^ 
be regarded as hypotheses to.be tested (McNeil and Popham, 1973, p. 241)." 
The results of our research efforts to da-te appear' clear: inconclusive. 

Where do we go from here? Should we continue to refine our instru- 
— ^ ments and methodologies or do our underlying concepts and rationals 

require re-examination? A recent article by Snow (1974) provides some 
> guidance for new te.search designs that may help overcome some of the - 
technical problems. 

Sanders (1972) provides a fresh insight into our conceptual 
difficulties by suggesting that, "the fault perhaps lies^not with the 
Inadequacy of the proposed answers or solutions, but rather with the 
inadequacy of the proposed questions (p. 12)." He goes on to suggest 
that teaching compiitence may not have\a core of essential attributes; 
that it may be a disjunctive rather than a conjunctive concept. Certainly 

the perspective oife has plays a major role in the type of research studies 

% ' 
' % , * ' 

that are carried out. 

Andther, equally basic, question about teacher evaluation' may be 

asked from a decision theory point of view. The question is WHY? Why 

evaluate teachers (or teacher-trainees)? What decisions are going to be 

made as ^ result the evaluation? What types of information are essential 

for a particular type of decision? More leading questions might include 



\ 



0 ) 

ERIC . 8 / 



*'Are we cpllectlng data to slustlfy political decisions?" or •'Do we 

collect data so that we can say we collect data?" It is this perspective 

of "why" that underlies the rema'liider of this article. 

Scriven (jL^74) clarifies the distinction between goals of 

evaluation (to answer certain types of question about certain entities) 

and Che roles of evaluation (e.g. part of a teacher training activity; 

curriculum development; investigation into purch^e of AV equipment). 

Scriven goes on to stat<^ his now well known definition of formative 

'evaluation: Formative evaluation plays "a role in the on-going improve- 

ment of the curriculum (p.62)," He then focuses in on the heart of the 

problem, "By stressing the constructive part evaluation may 'play in 

hon-threatening activjjties (roles) we slur over the fact that its goals 

always include the estimation of merit, worth, value, etc. which all. 

too clearly contributes in another role to decisions about promotion 
\ • 

and 're jection of personnel and courses (p. 63)." 

SUGGESTJION 

It is easy to conceptualize ^ varriant of formative evaluation, 
that refers specifically to the role of on^i^oing improvement of teacher 
effectiveness. Ideally, it would be desirable If this primary role^ere 
supplemented With anothe^^role - that of maintaining or improving 
general staff morale. 

At this stage, another useful distinctibn needs to be made - 
that of the level of teacher activity under 'consideration, *A molecular 
level might refer to proce'fesLes and results that occurs within a class- 
room setting whereas a molar level may refer to one's overall workload 
and the attendent results. It seems to be a fair statement that almost 
all research activities to date have concentrated on the formeY level - 

' . 9 



a natural emphasis when one is trying to discover relationships between 
teaching and learning - whereas many administrative concerns seem to ' 

r ■ 

emphaslz'e a more molar IBVel of activity. 

During tTie last d(6cade industrial, coiranerclal and government 
agencies have shown increasing interest in^a managerial process referred 
to as Management By Objectives (MBO) . This process appears to be an 
example of such a molar approach to evaluation* The essential components 
of MBO are, (1) the setting of objectives, (2) some form of review of 
progress toward objectives, (3) some appropriate action and (4) t^^ ^ 
setting of new objectives. MBO may play many roles: planning, apprjpsal, 

determination of rewards and increasing motivation. With^respect to this 

. ' ' ' ■ 'i ■ • 

latter role, Reddin (1971) says, **If properly introduced, MBO has a 
high motivational content, especially if objectives are mutually set 
^with sup*^iors, if the superior is seen more as a coach and less as a ^ 
judge, and \t those who consistently perform well are ultimately rewarde 

(p. 17)'.:: 

. Although it is not labeled as MBO, an example of this. perspective 
in an educational setting Is provided by Harcleroad (1971) :^ Accreditation 



by thts association (Northwest Association) is based on the institution's 



total strength and^^^^particiflar upon the success ol the institul^n' 
and each of its constituent par^s^-djci^^^ormulatittg and accomplishing Its 
specific objectives. The clarity of institutional objectives and the 
effectiveness of organization and alteration in the attainment of these 
objectives are of chief concepfx in final appraisal (p. 7)." 

Herman (1973) ppcr^des a complete example such a plan for 
evaluating teachers, a(dministrators , custodians, secretaries, and cafeteria^ 
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employees. Many of the^ ideas and examples in his book have been incorp-^ 
orated into the Faculty Performance Objectives Form (Appendix" I) de- 
signed for the Faculty of Education at Queen's University. This latter^ 
* " • rfF- . 

form is based on the^ juxtaposition two concepts: (i) a detailed 

♦ ♦ ^ ' ' / * , 

descrlptioiv of "one'a anticipated workload fdr a twelve-month period 
and (2) explicit* provision for mid-term -and end-of-term evaluat^^ 
o^ achievement as compared with anticipated achievement. The form has 
been specifically t^ailfcred to^ the program offered at Queen's University, 
but it may be easily modified by appropriate inser^on and deletions 

to meet the requirement of other . institutions. 

■ *^ »• 

Referring to Scriven's distinction between goals and roles of 
evaluation, the Faculty Performance Objective Form has as its goal a 
complete and accurate description of one's activities, and the necessary 
judgement of .J:he value of these activities. The achievement of this 
goal would appear to be a marked* improvement over more heuristic atafe- 
raents of one^s individuftJ- performance. 

t 

With respect to evaluation roles for this instrument, three pos- 
sibilities immediately come to mind:Al) useful for overall and specific 
planning, (2) additional form of motivation for individual faculty 
members and (3) source of information for promotions, tenure and firing. 
' The general thesis of this report has been to teview current. 



approaches to teacher eyaluation, to identify the molecular level of 
most such fnvesfrigatidns, to Suggest 'that a more molar approach may be 
more appropriate within some faculties and to identify one such approach 
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Since this form Is relatively detailed and lon^(15 pages) , 
a two~page Summary Sheet (Appendixffl) has Been designed to provide 
a profile of an individual's performance. As an example, a senior 
administrator and the faculty member could sit down together and, using 
the main form as a reference, rate the individual/ s contribution on 
each of the categories- A quick scan of the resulting profile should 
indicate areas of emphasis. It would also be a relatively easy manner 
to summarize the results of all of these individual profiles to obtain 
an institutional profile, 

4 
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Cemment (Dec. 31) 



TEACHING (B.Ed, program ) 

B.Ed, courses 'offered for first time 



No 
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Hr/Wk 


Wks. 


No students. 
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> offered 
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again 
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FACULTY PERFORMANCE OBJECTIVES 
DESCRIPTIOI^ (July 1) Comment (Dec. -i^l) 



TEACHING (M.EC program ) 
M.Ed. courses offered for first time 
No • Title No of students' 



M.Ed, courses offered again fe/at, substantially 
revised 



'No ' 


Title 


No of students. 




/ 























M.Ed, courses offered again 

No Title No of students 




Comment (May 30) 











—J 1 


= ; 

/ 
/ 








« 


, — 1^1- 


~^ 









Input into other M.Ed, iourses 

No Title Alo of students 
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FACULTY PERFORMANCE OBJECTIVES FORM 



DESCRIPTION (July 1) 



Comment (Dec 31) 



Comment (May 30) 



Continuing Program Advisor for 



1 

hum 



New Program Advisor for 
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FACULTY PERFORMANCE OBJEeTl\^ES. FORM 
DESCRIPTION (July 1) - Come/t (Dec.\31) 



Project Supervisor for 
Name ' 



Title 



Second reader for Project for 
Nam^ • Title 



Thesis chairman for 



Name ' * 



Title 



Thesis committee member for 



Name 



Title 
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FACULTY PERFORMANCE OBJECTIVES FORM 



DESCRIPTION (July 1) 



Coiment (Dec. 31 ) 



Coitmefit (May 30) 



TEACHING (ContHiuinq Education ) 



Workshops offered 



Title 


Hours 

V 


No of students 














Workshop parti 
Title 


• 

cipant 
Hours 


~ — r 

No of students 

r 




-y 











Public speaker for 

Title Purpose > 
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FACULTY PERFORMANCE OBJECTIVES FORM 
•DESCRIPTION (Jjuly 1) Comment (Dec. 31) 



SCHOOL ACTIVITIES 
Student Teaching 



School Liaison for 



Meetfftgs with teacher's/administrators 



Comment (May 30) 
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FACULTY PI 



DEScApTION (July 1) 



RFORMANCE .OBJECTIVES FORM 
Coninen;if (Dec. 31) 



Comment (May 30) 



Meetings with Board/Ministry officials 



Ac'ti vi ti es - i nVol vi ng students 
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FACULTY PERFORMANCE OBJECTIVES FORM 



DESCRIPTION 




Comnent (Dec. 31) 



Comment (May 30) 



"7^ 



• PROGRAM DEVELOPMEN T 
Discussions 



Submitted Proposals 



Acti vi ties 
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FACULTY PERFORMANCE OBJECTIVES FORM 



DE~SCRIPTIO^uTy I) 



Comment (Dec. 31) 



Comraeht, (May 30) 



RESEARCH 
Discussal ons 



Submitted Proposals 



Activities 
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FACULTY PERFORMANCE OBJECTIVES FORM 



' DESCRIPTION (July 1) 



Comments- (Dec. 31 ) • Comments (May 30) 



REPORTS/PUBLICATIONS 



B'ooks 



Reports 



Articles 



X 
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FACULTY PERFORMANCE OBJECTIVES FORM 



DESCRIPTION (July 1 ) 



Comments (Dec. 31) 



Comment (May 30) 



5 Corltinuinci Program AdVisor for 



New Program Advisor for 
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\ FACULTY PERFORMANCE O^Tl'vES^ FORM 
DESCRIPTION (July 1 ) . Comment (Dec. 31) 



Comment (May 30) 



Proiect Supervis.or for 
Name 



Title 



Second reader f-or Project for 

Name Title 



Thesis chairman for 
Name 



Title 



Thes'is comnjittee member for 



Name 



Title 
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FACULTY. PERFORMANCE OBJECTIVES FORM 

■ • ■ 








DESCRIPTION (July 1) 


Comments (Dec. 


31') 




Comments (May 30) < - 


, \ ■ — : — ' ■ ^ [ 










V- CnNSllLTING 










Meetings 'wi th - . ' 1 ' 

• 

: * , k 
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Member of project for 




















PROF&SSIONAL ACTIVITIES 
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Membership' in 
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Special Events/Tasks 
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FACULTY PERFORMANCE OBJECTIVES FORM 
DESCRIPTION (July 1) ' ' Comments (Dec. 31) 



Comments (May 30) 



commlTtee work : 

Facul ty " - 



University 



Other 
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n FAfc^LTY PERFORMANCE OBJICTIVES FORM " 
DESCRIPTION* (July 1) * Comments (Dec. 31) 



P 
■ft 



SELF-IMPROVEMENT 
Credit courses talcen 



Non-credit CQurses taken 



Workshops attended 



Conferences/Conventions attended 



Additional activities 
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Comments (May 30) 



FACULTY PERFORMANCE OBJECTIVES FORM 



DESCRIPTION (Jijl^ 1).. 



Comments (Dec. 31) 



Comments (May 30) 



PERSONAL PRIORITIES. 
■1. 



3. 
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FACULTY PERFORMANCE OBJECTIVES FORM 



-t 
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JMMARY SHEET 



Name; 



Date: 



DESCRIPTION 



TEACHING (M.Ed, program) 

1. M. Ed. courses first time 

2. M. Ed. courses revised 

3. M. Ed. courses repeated 

4. Input into other .courses 
'5. Continuing program advisor 

6. New program advisor 

7. Project supervisor 

8. Second project reader 

9. Thesis chairman 
10. Thesis committee / 



0 1 



TEACHING (.B.Ed, program) 

1. B. Ed. courses firsttime 

2. B. Ed. courses rey^sed 

3. B. Ed. courses repeated 

4. Input into other coursejs 

5. B. Ed. student 'advisor 



TEACHING (Continufng Education) 

1. - Workshops offered 

2. Workshop participant 

3. Rubllc Talks 



SCHOOL ACTIVITIES • 

1 . Student Teaching 

2. School Liaison 

3. Meetings with teachers^/administrators 

4. Meetings wi th' Board/Ministry officials 

5. Activities involving students 



PROGRAM DEVELOPMENT 

1. Discussions 
Proposals 



2. 
3. 



Activi ties 
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FACULTY PERFORMANCE OBJECTIVES FORM 



DESCRIPTION 


♦ 

0 \ .2 3 + • 


RESEARCH ' ■ ' • 

L. Discussions. ' * 

2. Proposals 

3. Activities; 




REPORTS/PUBLIGATIQNS t 

' 1 . Books 

Reports 
3. Articles ' ' 


* 


CONSULTING ' ' 

1. Meetings ^ 

2. Member , . 




PROFESSIONAL ACTIVITIES 

1. Membership ^ 

2. Special Events/Tasks 




COMMITTEE WORK 

V. Faculty. > 

2. University 

3. Other 

, t 


r ' ■ . 


self-Improvement 

1 . Credit courses^ • * 

2. Non-credit courses' 

3. Workshops V 

4. Conferences/Conventions . 

5. Additional activities • ' 

. ^ \ * . . 
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